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Abstract 

The invention of computer-based writing program has revolutionized the way of teaching 
second language writing. Embedded with artificial intelligence scoring engine, it can provide 
students with both immediate score and diagnostic feedback on tbeir essays. In addition, some of 
such programs offer convenient writing and editing tools to facilitate students’ writing process, 
such as My Access. However, the effectiveness of using such programs to improve students’ 
writing skills in the L2 is not conclusive yet. To gain a deeper understanding of how such a 
computer-based writing program works in EEL writing classes, this study explored factors that 
may lead to facilitation or frustration when the program My Access is in use. The participants 
were three EEL writing classes consisting of 68 third-year English majors in a Taiwan’s 
university. Data included students’ responses to a questionnaire, writing samples, feedback given 
by My Access, and focus-group interviews with the students. The findings show that only 55% of 
the students felt this program either moderately or slightly helpful to them in improving their 
writing skills and that the students were much less satisfied with the grading function than the 
writing/editing functions. Their dissatisfaction was largely because this program failed to give 
specific feedback in the content and rhetorical aspects of their writings. Moreover, the study also 
found that the different ways the three writing classes used this program and the instructors’ 
familiarity level with the program coupled with their technological skills greatly affected the 
students’ attitudes toward this program. The study concludes that pedagogical approaches and 
contextual factors are more crucial than the functions of the program in determining its 
effectiveness. 



INTRODUCTION 

The rapid development of computer technology, particularly artificial intelligence (AI), 
has given rise to a brand new type of computer applications in the field of second language 
writing: computer-based writing programs. These programs, usually equipped with an 
automated essay evaluation tool, have been increasingly used as an alternative tool for 
improving students’ writing skills for two major benefits. First, they provide immediate 
feedback from reports on grammatical errors to holistic evaluation of the content and 
organizational aspects of essay writing; second, they provide convenient writing and editing 
tools to facilitate students’ writing processes. Moreover, these programs also help writing 
instructors reduce time spent correcting and commenting on students’ compositions (Warden 
& Chen, 1998). Thus, writing instructors can have more time to turn their attention to other 
aspects of process-oriented writing instruction (Chen, 1997). 

Among many computer-based writing programs. My Access is probably one of the most 
popular implemented by many universities in Taiwan to improve students’ writing .skills. 
MyAccess functions as both a writing tool and an essay grader. As a writing tool, it provides 
scoring rubrics, online portfolios, word banks, a thesaurus, and an editor including checkers 
for spelling, grammar, punctuation, and word usage. As an essay grader, it provides an 
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automated holistic score on an either 4-point or 6-point scale along with immediate 
diagnostic feedback in terms of focus, development, organization, style, and grammar and 
mechanics. Hence, students can find out their strengths and weaknesses and focus more on 
particular domains in which they receive lower scores. Some research has shown that 
MyAccess is a useful tool for students to write better revisions (Elliot & Mikulas, 2004). 
However, other studies have found that the automated feedback provided by this program is 
insufficient and unspecific, thus providing little help in students’ writing process and 
sometimes even causing frustration (Yu & Yeh, 2003; Yang, 2004). 

The effectiveness of using such a computer-based writing program to improve students’ 
writing skills is not conclusive yet. More importantly, though many studies have evaluated 
various functions of these programs, we do not know clearly how they are actually used in 
the writing class, how they are integrated into the writing curriculum, for what pedagogical 
purpose they are used, and what attitudes both writing instructors and students have towards 
the programs. There may be many pedagogical factors other than the design of the program 
itself that will affect its effectiveness. To gain a deeper understanding of how such a 
computer-based writing program works in EEL writing classes, this study aims to explore 
factors that may lead to facilitation or frustration when the program is in use. 



LITERATURE REVIEW 

The interest in computer-based writing programs, particularly those with automated 
essay evaluation tools, has been rapidly growing in the last ten years. These programs have 
been promoted as “cost-effective ways of replacing or enhancing direct human input” (Ware 
& Warschauer, 2005, p. 5). Two of the most well-known programs of this type are probably 
Criterion (developed by the Educational Testing Services) and MyAccess (developed by 
Vantage Teaming). Using these programs, students can choose from a range of practice 
essays topics to write multiple drafts and receive immediate feedback in the form of both 
holistic scores and diagnostic comments on grammar, organization, style and usage. The 
electronic graders of these programs have been trained to look for lexical complexity, 
syntactic variety, topical content, and grammatical errors based on a set of previously scored 
responses containing "known score" papers marked by human graders for each score point. 
These marked papers are used as a basis for the programs to infer the rubric and the pooled 
judgments of the human graders. The agreement rate on holistic scores between the Criterion 
e-rater and human expert graders is typically 97% (Burstein, Chodorow, & Eeacock, 2003). 

As for MyAccess, its scoring engine, IntelliMetric, has been claimed to achieve a 99% 
agreement rate by Elliot (2001), the Chief Operating Officer for Vantage Teaming. 

In addition to the reliability of automated scores, Higgins et al (2004) examined 
Criterion for its responsiveness to coherence in 989 twelfth-grade students’ essays. They 
found that Criterion is capable of identifying four aspects of coherence of essays, including a) 
relationship to the prompt (essay questions topic), b) relationship of other discourse elements, 
c) relevance with discourse elements, and d) error in grammar, mechanics, and usages. 
Moreover, Attali (2004) evaluated Criterion for its feedback and revision features. Using a 
large-scale field data from approximately 9,000 drafts and revised compositions of 
sixth-twelfth grade students, he found that the students understood the feedback given by the 
program and utilized the feedback to improve their revised versions, thus yielding better 
quality writings in terms of organization, development, grammar, word usage, and style. 

In a similar vein, several researchers are also interested in examining the efficacy of 
MyAccess. Based on four separate studies, Elliot & Mikulas (2004) reported that 
fifth-eleventh grade student writing skills, as measured by student performance on statewide 
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writing assessments, were significantly improved by using MyAccess. In a survey 
administrated to 94 eighth-grade students using this program, over 80% rated their 
satisfaction with the automated feedback on their essays as both helpful and accurate. 
However, Herrington (2001) found that the scoring engine of MyAccess can be fooled by the 
writer, partly because it awards scores for essay length, even when the content is less than 
adequate. One way of fooling the system is to copy the same paragraph many times and paste 
it within the composition. The writer can thus devise means of beating the program, rather 
than making genuine efforts to improve his or her writing. He also noted that the system 
failed to recognize nuances such as sarcasm, idioms and cliches used in students’ essays. 

Although a number of studies have shown the usefulness of the automated feedback on 
essay writing, many developers of such programs suggest that the automated essay grading 
tool should be used only as a supplement to classroom instruction (Burstein, Chodorow, & 
Leacock, 2003; Burstein & Marcu, 2003). Consonant with this viewpoint, Oladejo (2005) 
further points out that language teachers should not assume that such automated grading tools 
can, or will, replace human graders, for these programs can only detect the surface errors of 
students’ writing but fail in examining the content or rhetorical aspects of students’ writings. 
In addition, human graders outperform electronic graders because they can provide 
post-grading consultation to individual students, which gives the teacher an opportunity to 
clarify unclear comments or to explain unlearnt language forms. This kind of rich negotiation 
of meaning in the interaction can never be done by an electronic grader. 

In Taiwan, a number of studies have been conducted to investigate the effectiveness of 
MyAccess used in the EFL university context. Yu & Yeh (2003) examined the use of the 
computerized feedback of MyAccess along with a bilingual concordancer as facilitating tools 
in enhancing college students’ writing in English. They found that most students considered 
feedback from MyAccess useful for writing their first revisions, yet they soon felt little help 
after receiving similar or the same feedback several times. Some students also pointed out 
that the computerized feedback was not specific enough to help them revise their writings. 
Yang (2004) investigated the effectiveness of using MyAccess in three different contexts. Her 
study involved around 300 subjects from freshman English classes, English composition 
classes, and one class from a self study program. She found that, although a majority of the 
students felt positive toward the automated essay grading tool, particularly in terms of the fast 
speed of the feedback, they also pointed out that the fixed, repeated feedback became 
meaningless to them. It is also noteworthy that only 13% of the students thought that the 
scores they received from MyAccess were adequate, whereas more than a half felt uncertain 
about the scores. The students commented that, in comparisons with the feedback obtained 
from their writing teachers, the automated feedback seemed too general and inaccurate. Some 
of them even expressed that they did not trust the computer scoring. 

The main value of the above two studies was their focus on students’ perspectives of 
MyAccess used in the EFE university context. However, their findings did not provide 
sufficient insight into the reasons behind students’ perceptions in the use of this 
computer-based writing program, and neither did they describe how this program was 
actually used in the writing classes or how it was integrated into the writing curriculum. The 
present study, therefore, was conducted to gain a deeper understanding of what pedagogical 
factors, in addition to the design factors, may affect the effectiveness of using this program as 
both an essay grader and a writing tool in EFE college writing classes. 
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METHODOLOGY 



The Context 

MyAccess was put to use in three third-year EFL writing classes offered to English 
majors at a national technological university in the spring semester of 2003 academic year. 

The main purpose of implementing this program was to enhance students’ writing ability and 
lessen writing instructors’ workload. These three writing classes were taught by three 
different instructors and varied slightly in class size: 26 students in Class A, 19 in Class B, 
and 23 in Class C. The three writing instructors shared some commonalities in their teaching 
approaches. First, their course objectives all aimed to familiarize students with the essentials 
for writing academic essays. Second, they used the same textbook and taught similar contents. 
Third, they adopted a similar process-oriented writing curriculum. However, the way each of 
them used MyAccess in the class was different, though they all attended a one-hour training 
workshop provided by a MyAccess program trainer. The differences in the use of the program 
among the three instructors are reported in the next section as part of the findings, so that we 
can see how the program was used affected students’ reactions toward it. 

Data Collection and Analysis 

The data included the students’ responses to a questionnaire made by the researchers, 
three focus group interviews with the students from each writing class, and the students’ 
writing samples. The questionnaire surveyed students’ views and reactions concerning the use 
of MyAccess in helping them improve their writing ability, containing multiple choice 
questions and open-ended questions. In total, 53 out of 68 students (21 from Class A, 18 from 
Class B, and 14 from Class C) responded to the questionnaire. Their responses were coded 
first and then analyzed both quantitatively and qualitatively. 

To triangulate the findings from the questionnaire responses and uncover hidden factors 
affecting students’ perceived effectiveness of this program, focus group interviews were 
conducted as well. 5 students from Class A, 5 from Class B, and 6 from Class C agreed to 
participate in the interviews. Each focus group interview lasted approximately one hour. The 
interviewees were asked to talk about how MyAccess was used in their writing class, how 
they felt about MyAccess used as a writing tool and an essay grader respectively, and what 
problems they encountered when using this program. All the three focus group interviews 
were audio-taped and then transcribed in Chinese and translated in English. 

The third data source was students’ writing samples documented in MyAccess online 
portfolios. The online portfolios contained students’ writing records, including multiple 
versions of their essay writings along with each version’s scores and diagnostic feedback 
given by MyAccess. A pseudonym was given to each student for the sake of privacy 
protection. The collected writing samples were used for close examination to triangulate the 
findings particularly from the interview data. 

FINDINGS AND DISCUSSION 

Part A. Students’ Reactions toward MyAccess as an Essay Grader 

As shown in Table 1, the students’ reactions MyAccess as an automated essay 

grader were generally negative. There was only one item viewed positively by a majority of 
students (71%) - the speed of the feedback. It was disappointing to find that only 6% of the 
students felt satisfied with this grading function and over 50% indicated that they were not 
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satisfied. More surprising, none considered the scores given by this electronic grader 
adequate and over 60% regarded inadequate. Moreover, only one-fourth thought that the 
diagnostic feedback given by MyAccess helped them to revise their writings, but close to 50% 
found it not helpful at all. If we look at the mean of each item, the first three items did not 
reach 3.0 (Note that 3 indicates neutral), which revealed that the students’ overall reaction 
toward the grading function of the program, including automated scoring and diagnostic 
feedback, was quite unsatisfied. 



Table 1. Students’ Reactions toward MyAccess as an Essay Grader 



Item / (N=53) 


Agree 


Neutral 


Disagree 


Mean* 


1. I’m satisfied with the automated 
grading system of MyAccess. 


3 (6%) 


20 (37%) 


30 (57%) 


2.28 


2. The scores given by the grading 
system are adequate. 


0 


20 (37%) 


33 (63%) 


2.26 


3. The written feedback given by the 
grading system is helpful for revision. 


13 (24%) 


15 (28%) 


26 (48%) 


2.57 


4. The speed of the feedback is fast 
enough. 


38 (71%) 


13 (24%) 


2 (5%) 


3.87 



* Note: The mean is calculated based on a scale ranging from 1 to 5 (1: strongly disagree, 2: disagree, 3: 



neutral, 4: agree, and 5: strongly agree). 

Two important reasons that were identified from the students’ self-reports can explain 
their dissatisfaction with the program’s grading function: 1) fairness of the automated scores 
and 2) informativeness of the diagnostic feedback. 

1) Fairness of the automated scores 

Many students remarked strongly that the scores that they received from the automated 
grading system of MyAccess were unfair, thus inadequate. They did not think that the scores 
given by this electronic grader reflected their actual English writing ability. Here are a few 
examples of their comments: 

“The scores are not fair! The grading machine gives a score to your writing based on the 
quantity instead of the quality. That is, the more you write, the higher score you get.” 

“I remembered one time when I just finished the introduction, I clicked on the button of 
submission. To my surprise, I got 5^ for that one-paragraph writing. It was weird that a 
writing without a body and a conclusion still got a score of 5. Erom then on, I started to 
doubt the fairness of this grading function.” 

“The grading program was too kind to me! I mean sometimes I still can get high scores 
without adding a conclusion in my essays. I wrote only the introduction and the body, 
and the program gave me a score of 5. How come an essay without a conclusion can get 
such a high score?” 



* MyAccess provides a holistic score on a 6-point scale. A score of 6 indicates that the writer communicates his 
or her message very effectively and a score of 1 indicates that the writer does inadequately. 
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“The program likes to see we use many transitional phrases in our writing. For example, 
if I add “as a result” or “on the other hand” at the beginning of some sentences but 
without changing anything else, I can immediately gain one more point. That is one way 
we often use to get a higher score. See, we know how to trick the program!” 

“When I submitted the same piece of writing to both my writing teacher and MyAccess, I 
found that there was a big gap between the grades given by my teacher and those given 
by the software, so I don’t really trust the program’s grading function.” 

From the students’ comments, we can find that there are some flaws of the design in the 
scoring engine of MyAccess. It seems that the scoring engine favors lengthiness of an essay 
and the use of transitions, but it ignores the content and organizational aspects of writing at a 
discourse level. As Herrington (2001) pointed out, the scoring engine of MyAccess can be 
fooled by writers once they know the tricks. Thus, the students could use some tricks to gain 
a higher score for their writing although the ideas in their content were not well organized. 
This is a very important reason that made them doubt the fairness of the scores given by the 
program. The grading system does not appear to have the ability to judge whether the ideas 
presented in students’ writings are developed coherently and logically, though it claims to 
help students develop writing ability in the domain of “Content and Development.” 

2 

Below is a sample of a student’s essay (see Figure 1) that received a holistic score of 5 
from the grading system but failed to achieve coherence and completeness. As we can see, 
this essay contains only two paragraphs but no conclusion. We are not sure if the first 
paragraph can be called an introduction because it already gives many details describing 
some problems that people living in genteel poverty may have and some consequences these 
people may get. Moreover, this paragraph is not well organized, for it contains more than one 
controlling idea and some of the ideas presented here are not coherently or logically 
developed (For example, how come a person living in genteel poverty can “spend a lot of 
money”? and what does “people” refer to in the sentence “people’s lives will be awful”?). In 
addition, we can see that the writer mentions at the end of the first paragraph that genteel 
poverty “can bring about two major problems in our society,” but there is only one problem 
described in the second paragraph. This problem, as stated in the first sentence of the second 
paragraph “First, people are not strong enough to face the reality”, however, dose not directly 
point out the negative influence that living in genteel poverty may exert on our society. Also, 
the focus of this paragraph seems to be shifted to finding other means instead of work to get 
rich. There are also too many ideas in this paragraph and some of them are not connected 
well, particularly as shown in the last two sentences, which appear abruptly and have no clear 
connection to the previous sentences. 

Such an essay is likely to receive a low score if evaluated by a writing instructor, for it 
contains serious logical and organizational problems. In such a case, if students are satisfied 
with the score of 5 (because it is the second highest) and do not continue revising their works, 
then this program cannot provide them much help. Perhaps as one student describes, this 
grading system is “too kind” to them. Receiving a high score for such a draft, on the one hand, 
can help them build more confidence in their writing; on the other, it may also confuse them 
and hinder their continuous improvement. 



^ Although this essay received a holistic score of 5, it got different scores in specific domains: focus and 
meaning: 5, content and development: 4, organization: 4, language use and style: 4, and mechanics and 
convention: 5. 
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Figure 1. A Student’s Writing Sample 



Prompt: 

In our society we face a number of problems, such as crime, poverty, and pollution. What 
do you think is the biggest problem we face in our society today, and why is it a serious 
issue? Write an essay explaining the issue or problem you have chosen and why it is having 
such a negative effect on our society. 



Student’s Writing: 

People living in genteel poverty can be the biggest problem in our society. These people 
spend a lot of money to buy clothes, watches, and various ornaments to show off their fake 
social statuses. They hope that some precious and extraordinary opportunities will fall on 
them. Furthermore, they are busy applying for another credit card at the same time to 
prevent them from bankruptcy. As a matter of fact, they are unable to discharge their debts. 
In such a vicious circle, one day they may ruin our society and drag down our nation's 
economy. Definitely, people's lives will be awful. The significant causes can bring about 
two major problems in our society. 

First, people are not strong enough to face the reality. We work and we receive the rewards. 
However, some people do not work, but they reap the rewards as well because of their 
inheritance. Accordingly, this behavior forms an illusion that it is not a must to work for 
money. Most of people know what the truth is, but they lie to themselves. They would like 
to pay attention to their appearance instead of creating their own futures. They realize that 
they have to involve much cost to succeed, but they choose the other side around. A 
shortcut, which makes them feel rich, is to keep up appearance. After all, people who have 
awareness must coerce them to face the reality. Eventually, they will not only bankrupt but 
also impose a burden on all the citizens. 



2) Informativeness of the diagnostic feedback 

The second major reason why the students were not satisfied with MyAccess as an essay 
grader was that many of them felt the automated feedback was not specific or informative 
enough to guide them how to revise their writings; therefore, they still hoped to get individual, 
detailed feedback from the writing instructor. Furthermore, some found the feedback helpful 
only for their early drafts, but if their scores were not increased after revising their drafts, 
they would get similar or identical feedback, thus providing no help and even causing 
frustration because they had no clue how to strengthen their writings. Below are some of the 
students’ comments on the diagnostic feedback given by this program. 

“Basically, the feedback from MyAccess was quite general and vague. So, when I 
received such feedback, I still had no idea what my problem was exactly. I really hoped 
to get the teacher’s help to identify my personal weakness in writing.” 

“The first time when I got the comment from the system, I felt very excited and I found 
it was useful. However, I became more and more disappointed when I noticed the 
feedback each time I got was similar.” 
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Moreover, a few students pointed out that their essays were sometimes judged as 
“off-topic” by the system without any explanation, which made them even more frustrated. 
One student expressed her frustration and anxiety in the following statement: 

“I think I was a hard-working and responsible student, and I spent much time writing 
and revising my compositions. Unfortunately, it was often the case that my essay was 
judged off the topic, but I didn’t know why. Being rejected by the system meant that I 
got no point in my final grade. I was very upset and nervous about that!” 

This student felt frustrated and worried actually for two reasons. First, the program did 
not explain why her writing was off the topic and thus she did not know where to start 
revising. Second, the scores on each of her essays that she received from MyAccess were 
counted as part of her final grade in this course. That was a part of the grading policy her 
instructor set to “push” students to use this program, which made students care very much 
about the scores they got for their writings and hence increased their anxiety or even cause 
their frustration (We will discuss in more detail how instructors used this program differently 
later). Here we see that when the scores were emphasized by the instructor, the students were 
actually more eager to get specific diagnostic feedback from the program. 

Part B. Students’ Reactions toward MyAccess as a Writing Tool 

In regard to MyAccess as a writing tool, we investigated the helpfulness of the four 
important functions this program provides: My Editor, Thesaurus, Word Bank, and Online 
Portfolio. 1) My Editor is a proofreading system, which not only provides spelling correction 
but also analyzes text to detect and correct errors in grammar, mechanics, style, and usage. In 
addition, it provides an explanation of potential problem and suggestion on how this mistake 
might be improved. 2) The Thesaurus is an on-line dictionary offering a list of synonyms for 
the word being consulted, so that it can help students expand their vocabulary size and avoid 
the repetitive use of the same words in their writing. 3) The Word Bank offers words and 
phrases for a number of writing genres, including comparison, persuasive, narrative, and 
cause-effect types of essays. Eurther, it also offers words and phrases used for a summary and 
a conclusion. 4) On-line Portfolio allows students to document and access their own writings 
and revisions anytime they desire. It contains a student’s multiple versions of essays along 
with the feedback and scores given by MyAccess. 

Table 2 shows that the students generally did not think that these four functions 
facilitated their writing process to a considerable degree, for the mean of each item did not 
reach 3.0. My Editor was the one viewed slightly more helpful than the other three functions. 
Those who considered My Editor useful in facilitating their writing process reported that this 
function helped them easily detect their grammatical and spelling mistakes along with 
punctuation ones. They also added a remark that the frequent use of the grammar checker 
raised their awareness of their commonly made mistakes. 

Eor the Thesaurus and the Word Bank, only slightly over one-fourth of the students find 
them helpful. Some students pointed out that these two functions helped them expand their 
vocabulary repertoire and sometimes even stimulated them to come up with more appropriate 
words or phrases to express their ideas. Others did not find them useful because of two major 
reasons: first, the corpus size was not large enough; second, the two functions only listed 
words and phrases without providing examples of how they are used in sentences. They 
suggested that the program should expand the corpus size and provide examples illustrating 
the use of the listed words and phrases in the context. 




Table 2. Students’ Reaction toward My Ac cess as a Writing Tool 



Item / (N=53) 


Agree 


Neutral 


Disagree 


Mean* 


5. It is helpful to use My Editor during 
my writing process. 


17 (32%) 


17 (32%) 


19 (36%) 


2.85 


6. It is helpful to use Thesaurus during 
my writing process. 


14 (26%) 


19 (36%) 


20 (38%) 


2.81 


7. It is helpful to use Word Bank during 
my writing process. 


14 (26%) 


16 (30%) 


23 (44%) 


2.68 


8. It is helpful to use Online Portfolio 
during my writing process. 


8 (15%) 


22 (41%) 


23 (44%) 


2.53 



* Note: The mean is calculated based on a scale ranging from 1 to 5 (1: strongly disagree, 2: disagree, 3: 



neutral, 4: agree, and 5: strongly agree). 

Online Portfolio was the one viewed least helpful among the four functions. Several 
students who liked this function remarked that it helped them maintain and access their 
writings and feedback easily and conveniently. It also helped them constantly check their 
progress. When they found the essays they wrote were getting better and better, they gained a 
sense of achievement as well. Other students did not find this function helpful mostly because 
they seldom used it during their writing process. 

It is also noteworthy that approximately 40% of the students did not consider these 
functions helpful. These students either seldom used the four functions and thus did not see 
the values or they used these functions several times but found too limited. However, there is 
another factor that may not be identified immediately but can affect students’ perceived 
helpfulness concerning the use of this program: the way the writing instructor uses the 
program in the class. Next we will compare the use of the program in the three writing classes 
and discuss how pedagogical factors may affect the effectiveness of its use. 

Part C. Effectiveness of Using MyAccess in the Three Writing Classes 

When the students were asked to self-evaluate the effectiveness of using MyAccess to 
improve their writing skills, only 55% indicated that it was helpful, yet the helpfulness was 
either moderate or slight but not to a great extent, and the other 45% did not view it helpful at 
all (see Table 3). This result was not surprising when compared with the first two tables. 

Since the students generally held negative views on MyAccess as both an essay grader and a 
writing tool, it would be predictable that they did not consider it effective to use this program 
to improve their writing ability. However, it is of interest to find that students in each writing 
class perceived the effectiveness differently. 

Looking at both Table 3 and Table 4, we can find that the students in Class A appeared to 
view the program more positively than the students in the other two classes. For example, 
only 14% of the students in Class A thought that the program provided no help for their 
writing improvement, but there were 72% in Class B and 58% in Class C who denied its 
helpfulness. In addition, we can also find that Class B was often the group who viewed the 
program least favorably, yet they valued the writing tools offered by the program slightly 
more highly than Class C. 
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Table 3. Students’ Perceived Effectiveness of Using MyAccess - Class Comparison 



To what degree do you think using 
MyAccess can help you improve 
your writing skills? 


Greatly 

helpful 


Moderately 

helpful 


Slightly 

helpful 


Not 

helpful 


ClassA(N=21) 


0 


6 (29%) 


12 (57%) 


3 (14%) 


Class B(N= 18) 


0 


1 (6%) 


4 (22%) 


13 (72%) 


Class C (N=14) 


0 


2 (14%) 


4 (28%) 


8 (58%) 


Total (N=53) 


0 


9 (17%) 


20 (38%) 


24 (45%) 



Table 4. Students’ Reaction toward MyAccess - Class Comparison 



Item 


Mean* 


Class A 


Class B 


Class C 


I . I’m satisfied with the automated grading 
system of MyAccess. 


2.76 


1.78 


2.21 


2. The scores given by the grading system are 
adequate. 


2.43 


2.06 


2.29 


3. The written feedback given by the grading 
system is helpful for revision. 


2.71 


2.39 


2.57 


4. The speed of the feedback is fast enough. 


3.76 


3.56 


4.14 


5. It is helpful to use My Editor during my 
writing process. 


3.19 


2.72 


2.50 


6. It is helpful to use Thesaurus during my 
writing process. 


3.09 


2.67 


2.57 


7. It is helpful to use Word Bank during my 
writing process. 


2.95 


2.56 


2.43 


8. It is helpful to use Online Portfolio during 
my writing process. 


2.71 


2.39 


2.43 



* Note: The mean is calculated based on a scale ranging from 1 to 5 (1: strongly disagree, 2: disagree, 3: 



neutral, 4: agree, and 5: strongly agree). 

What made the three writing classes differ to a considerable degree in their perceived 
effectiveness of using MyAccess! Through the focus group interviews, we found out that, 
although the three instructors followed a similar process-oriented writing curriculum and 
used the same textbook, the ways they used this program in their classes were different, 
including the emphasis on the program’s functions, the requirements and the grading policy, 
the number of essays completed using the program, and the ways of post-grading feedback 
and consultation provided by them. In addition, the instructors’ familiarity with the software 
along with their technological skills may also affect how the program was used and how the 
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students felt about the helpfulness of its functions. Table 5 summarizes the differences in the 
use of this program in the three writing classes. 



Table 5. Different Ways of Using My Access in the Three Writing Classes 





Class A 


Class B 


Class C 


Ways of Introducing 
MyAccess 


Teacher gave a detailed 
instruction and in-class 
demonstration on how 
to use the program as 
both an essay grader 
and a writing tool. 


Teacher gave a brief 
instruction and asked 
students to explore the 
program by themselves. 


Teacher gave a detailed 
demonstration on how 
to use the grading 
function, but she asked 
students to explore the 
editing functions by 
themselves. 


Emphasis on the 
Functions of 
MyAccess 


Teacher emphasized 
both grading and 
editing functions. 


Teacher emphasized 
more on the various 
editing functions. 


Teacher emphasized 
more on the grading 
function. 


Requirements & 
Grading Policy 


Students were asked to 
write multiple drafts on 
assigned topics using 
the program at home. 
They were not allowed 
to turn in their essays to 
the teacher until they 
got a score of 4 given 
by the program. 


Students were asked to 
write paragraphs on 
assigned topics using 
this program in class. 
Then they were asked 
to develop paragraphs 
into an essay at home 
and submit it to the 
teacher. The scores 
given by the program 
were not important. 


Students were asked to 
write essays on 
assigned topics using 
this program at home. 
The scores they got 
from the program 
accounted for 40% of 
their final grades. 


Duration of Using 
MyAccess 


4 months 


1 month 


4 months 


Essays Completed 
Using MyAccess 


4 essays 


2 essays 


6 essays 


Post-grading 
Feedback & 
Consultation 


Teacher gave individual 
written feedback to 
each student essay and 
held class discussions 
on selected essays for 
each writing topic. 


Teacher gave individual 
written feedback to 
each student essay and 
provided after-class 
consultation for 
students when needed. 


Teacher gave brief 
written feedback to 
students’ essays at the 
end of the semester, but 
not right after they 
turned in their essays. 


Teacher’s 
Familiarity Fevel 
with MyAccess & 
Technological Skills 


High familiarity and 
advanced technological 
skills. 


Fow familiarity and not 
sufficient technological 
skills. 


High familiarity only 
with the grading 
function and advanced 
technological skills. 



When comparing the ways MyAccess was used in the three writing classes and the 
students’ reactions toward the program (see Tables 3, 4, and 5), we can infer that pedagogical 
factors play a significant role in affecting the effectiveness of using this writing software. As 
the results found in Class A reveal, the program seems to work the most effective when the 
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instructor asks students to use it as both an essay grader and a writing tool but only treat it as 
a facilitator and a supplement to classroom instruction. The program can help students write 
drafts in the right direction and reduce surface errors, but in terms of content development 
and organization, students still need their writing instructors to give them more specific, 
detailed feedback or provide post-grading consultation to further help them with their 
individual writing problems that the software cannot detect. On the other hand, we also see 
that the instructor’s over reliance on the grading function of the program without giving 
proper, timely assistance to students (as found in Class C) may cause confusion or frustration 
to students when the program is used to facilitate their writing process. 

Finally, the instructor’s familiarity level with this computer-based writing program along 
with their technological skills may also affect the effectiveness of using the program in the 
class. From the interview data, we found that many students emphasized that the instructor’s 
computer and technological skills needed to be competent enough to use a computer- assisted 
language learning program in the class. They further commented that the instructor’s 
unfamiliarity with the software and insufficient technological skills affected their attitudes 
toward the use of the program, as a student from Class B remarked, 

“It was totally a mess when we tried using MyAccess in the class for the first time. 

When my teacher demonstrated the program and taught us how to use it, she still 
encountered a lot of technical problems. At that moment, what I thought was - how 
could I use the software to help my writing if my teacher did not know how to use it 
well? So, I was not surprised to know that my teacher decided to stop using MyAccess 
after we tried two writing topics. We [the teacher and the students] all felt frustrated 
when we were using the program.” 

In fact, the students in Class B used MyAccess to facilitate their writing for only one 
month. The instructor and the students reached a consensus on discontinuing the use of it 
after they had tried writing two essays with the program, because they considered the 
interface of MyAccess not user-friendly and had too many technical problems that they were 
unable to solve. It was mainly the frustration from the technical aspect that made them give 
up using the program; accordingly, this group’s perceived effectiveness of using MyAccess 
was often the lowest in many regards. However, students in Class A and Class C did not 
report that they encountered many technical problems and even if they did, they could 
quickly solve the problems with the instructor’s help. 

It was also worth noticing that the students in Class C viewed the editing functions 
provided by MyAccess even more negatively than the students in Class B. This can be 
explained probably from the fact that the instructor in Class C did not introduce the various 
editing functions to the students and neither did she emphasize the advantages of using those 
functions. When the teacher is not familiar with the program’s functions and not aware of the 
values of those functions, it would be difficult for students to find out the values and make 
good use of these functions on their own. 

CONCLUSION 

This study is a small-scale exploratory one. The main purpose is not just to determine 
the effectiveness of using MyAccess in some EFL university writing classes in Taiwan, but 
more importantly, it is intended to find out factors residing both in the design of the program 
itself and in the pedagogical aspects of using this program that may affect the effectiveness of 
its use. In fact, as many researchers, educators, and developers of computer-based writing 
programs have recommended or even insisted, all computer-assisted learning programs 
should be used as a supplement to classroom instruction but never as a replacement of the 
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teacher. There is no single system or program that is perfect in all aspects; thus, we should 
avoid an unreasonable expectation that any good software should be able to do everything. As 
Bax (2003) warns us, people using computer technology for language learning and teaching 
often fall into two fallacies: the “Omnipotence fallacy” and the “Sole Agent fallacy” (pp. 
25-26); that is, they tend to assume that computers can do everything and should replace 
current learning and teaching technologies or even the teacher and that the key or only factor 
in successful implementation of the technology is the technology itself. We may often 
unconsciously exaggerate the power of technology but ignore the contextual aspects of how 
the technology is used and what pedagogical factors will affect the effectiveness of its use. 
This paper hopes to increase language teachers’ awareness of how such a computer-based 
writing program can be used more effectively as a facilitator in the writing class to benefit 
both students and the writing instructor. Even though we have found a number of weaknesses 
and limitations of the program itself, it does not mean that we should stop using it 
immediately. Instead, we probably need to reexamine the learning and teaching context and 
put its values into the best use and see if the program can help us meet particular contextual 
and pedagogical needs. 

There is another crucial issue worth our attention concerning the use of such a writing 
program: What is the goal for us to use a computer-based writing program to facilitate 
students’ writing? Or more specifically, what kind of writing do we wish our students to 
produce with the help of such programs? As Leki (2005) claims, the term “good writing” 
should be operationalized differently in different contexts and for different purposes. What 
good writing will look like and how good a student writer will be expected to become are 
imperative questions that writing teachers will need to ask themselves and decide what the 
answers will be. We cannot just claim that we are using a process-oriented approach without 
specifically defining what “good writing” is for our students to achieve. Ware & Warschauer 
(2005) point out that computer-based writing programs currently hold up rather static models 
of what is considered “good writing”, for example, a prototypical five-paragraph essay. They 
suggest that second language researchers will need to develop more rigorous evaluative 
criteria that reach beyond the formulaic models. Before such new writing technologies are 
developed, we need to re-consider whether such “static” writing models are the ones we want 
our students to learn and to produce. When their goal is to attain a high score on a large-scale 
institutionalized or commercialized writing test, programs such as MyAccess can probably 
provide them great help in training them to produce “good writing” that the test graders 
prefer to read. On the other hand, if the goal is to help our students to become more creative 
and imaginative in their writing, or as Leki (2005) said, to help them “to create L2 texts that 
come to accurately reflect their maturity and expertise” (p. 88), then such computer-based 
writing programs are probably not a good choice. This can also explain why students may 
feel a big gap between their instructor’s evaluative feedback and the one given by the scoring 
engine. The core issue exists in what model we are using for “good writing”. 

More than 35 years ago, Mesthene (1969) already stated clearly that “new technology 
creates new possibilities for human choice and action but leaves their disposition uncertain. 
What its effects will be and what ends it will serve are not inherent in the technology, but 
depend on what men will do with technology” (p. 492). For optimal learning and teaching to 
occur, we writing teachers, on the one hand, welcome new possibilities that new technology 
brings to writing instruction, but on the other, we need to make deliberate decisions on how 
to put new technology into more effective use in our particular context and on how not to use 
new technology for inappropriate or undesirable purposes. MyAccess is only one example of 
such new technologies, and it is often not the case that MyAccess itself that will lead to 
facilitation or frustration, but, more crucially, the way it is used by both the teacher and 
students determines how effective it can be. 
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